PCT 



WORLD INTELLECTUAL PROPERTY ORGANIZATION 
International Bureau 




INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) 



(51) International Patent Classification 4 ; 
H04L 11/20, G06F 15/16 



Al 



(11) International Publication Number : WO 86/ 0251 1 

(43) International Publication Date: 24 April 1986 (24,04,86) 



(21) International Application Number: PCT/US8 5/0 1839 

(22) International Filing Date: 26 September 1985 (26.09.85) 

(31) Priority Application Number: 661,995 

(32) Priority Date; 18 October 1984 (18.10.84) 

(33) Priority Country: US 



(71) Applicant: HUGHES AIRCRAFT COMPANY [US/ 

US]; 200 North Sepulveda Boulevard, EI Segundo, 
CA 90245 (US). 

(72) Inventor: McMILLEN, Robert, J. : 2508 East Willow 

#104, Long Beach, CA 90806 (US). 

(74) Agents: LINK, Lawrence, V., Jr. et al.; Hughes Aircraft 
Company, Post Office Box 1042, Bldg. C2 t MS AI26, 
El Segundo, CA 90245 (US). 



(SI) Designated States: DE (European patent), FR (Euro- 
pean patent), GB (European patent), IT (European 
patent), JP, KR, NL (European patent). 



Published 

With international search report. 



(54) Title: LOAD BALANCING FOR PACKET SWITCHING NODES 



22 

i 



21 



ENABLE - 



TAG- 



24 



QUEUE 
SELECTOR 



iux/ 

TJT 



OFFSET 
/ 28 



LOAD 
BALANCE 



T 



3^ 



£1 



26 
\ 



(57) Abstract 

A load balancing circuit ar- 
rangement for use with a packet 
switching node. The packet switching 
node processes applied data packets 
containing routing tag signals indica- 
tive of the output port destinations to 
which the data packets are addressed, 
and routes these packets to the ident- 
ified output ports. The present inven- 
tion a load balancing circuit coupled 
to the packet switching node which 
monitors the output port addresses of 
the applied data packets and monitors 
the number of data packets addressed 
to each of the output ports. The load 
balancing circuit is adapted to gen- 
erate new routing tag signals identify- 
ing output port addresses which redis- 
tribute the output port load. The load 
balancing circuit arrangement in- 
cludes a tag selection circuit coupled to the load balancing circuit and the packet switching node which selectively replaces 
the routing tag signals of the applied data packets with the new routing tag signals in order to redistribute and balance the 
output port load. The load balancing circuit comprises a minimum index circuit for generating the new routing tag signals 
and an adder circuit coupled thereto. The minimum index circuit combines the new routing tag signals with offset signals 
that modify the new routing tag signals in order to implement a predetermined output port priority scheme. The load bal- 
ancing circuit arrangement may be employed in both multiple queue and multiport memory packet switching nodes em- 
ployed in computer or telephone communications applications. 
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LOAD BALANCING 

FOR PACKET SWITCHING NODES 

BftCggRPPW OF TAB DISCLOSURE 
The present invention relates generally to 
packet switching nodes employed in multi-processor and 
parallel computer systems , and the like, and more 
particularly to a load balancing circuit arrangement 
for use in such packet switching nodes which redistrib- 
utes incoming data packets to its output ports for more 
efficient processing* 

One developing area of computer technology 
involves the design and development of large-scale, 
multi-processor-based distributed and parallel computer 
systems* Typical of these classes of computer systems 
and architectural approaches are the single instruction 
stream, multiple data stream (SIMD) computer architec- 
ture and the multiple instruction stream, multiple data 
stream (MIMD) computer architecture. 

A SIMD computer typically comprises a control 
unit, N processors, N memory modules and an intercon- 
nection network. The control unit broadcasts instruc- 
tions to all of the processors, and all active proces- 
sors execute the same instruction at the same time. 
Each active processor executes the instruction on data 
in its own associated memory module* The interconnec- 
tion network provides a communications facility for the 
processors and memory modules* 
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1 A HIHD computer typically comprises H proces- 

sors and N memories, and each processor can execute an 
independent instruction stream. Each of the processors 
may communicate to any other processor* Similar 

5 interconnection networks may be employed in the MIMD 
computer - 

Various interconnection networks may be 
employed to interconnect processors and memories 
employed in either type of computer system. These 

10 interconnection networks include delta networks, omega 
networks, indirect binary n-cube networks, flip net- 
works, cube networks and banyan networks, for example* 

The above-cited networks are discused in some 
detail in the following publications: "LSI implementa- 

15 txon of modular interconnection networks for MIMD 

machines," 3,980 Tnt'l. Conf. Parallel Processing, Aug. 

1980, pp. 161-162? "Analysis and simulation of buffered 
delta networks ," IEEE Tra ns. Computers, Vol. C-30, pp. 
273-282, April 1981; "Processor-memory interconnections 

20 for multiprocessors," gth Annual Int'lt gyflp* Computer 
Architecture . April 1979, pp. 168-177; "Design and 
implementation of the banyan interconnection network in 
TRAC," AFTPS 1980 Wafc'l. Computer Conf., June 1980, pp. 
643-653; "The multistage cube: a versatile interconnect 

25 tion network," Computer . Vol* 14, pp. 65-76, Dec. 1981; 
"The hybrid cube network," Distribu ted Data Acquisi- 
tion, Computing a nd Control Svmo., Dec. 1980, pp. 
11-22; and "Performance and implementation of 4x4 
switching nodes in an interconnection network for 

30 PASM," 1981 Tnt'l Conf. on Parallel Processing, Aug. 

1981, pp. 229-233. 

Several types of data switching techniques 
may be employed to transfer data in SIHD and MIMD 
computers, and the like, including packet switching, 
35 message switching, time-division circuit switching or 
space-division circuit switching. Packet switching 
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1 involves sending one or more words of data at time 
through the system . 

A multiple queue packet switching node is de- 
scribed in a presently copending patent application 

5 entitled "Packet Switched Multiple Queue NxM Switch 
Node and Processing Method 9 m invented by R. J. 
McMillen, and assigned to the assignee of the present 
invention. This patent application discloses a packet 
switching node which processes applied data packets 
10 containing routing tag signals indicative of the output 
port destination to which the data packets are to be 
applied and transfers these packets to those output 
ports • 

The packet switching node comprises a plural- 

15 ity of input ports and a plurality of output ports. A 
plurality of queue selectors are individually coupled 
to corresponding ones of the plurality of input ports. 
Each of the plurality of queue selectors are adapted to 
route data packets applied to each of the input ports 

20 in accordance with the output port destination of the 
data packets. 

A plurality of queue sets are individually 
coupled to corresponding ones of the plurality of queue 
selectors. Each of the queue sets comprise a plurality 

25 of queues for storing and forwarding data packets 

applied thereto as a function of output port destina- 
tion. A plurality of output arbitrators are individu- 
ally coupled between corresponding ones of the plurali- 
ty of output ports and the respective queue of each of 

30 the queue sets which store and forward data packets 

whose destinations are the corresponding output port. 
The output arbitrators are adapted to transfer the data 
packets stored in the queues to the corresponding 
output port in accordance with a predetermined priority 

35 arbitration scheme. 
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1 Another related presently co-pending patent 

application is entitled "Packet Switched Multiport 
Memory NxM Switch Node and Processing Method," invented 
by R, J. HcMillen and A. Rosman, and assigned to the 
5 assignee of the present invention. This packet switch- 
ing node processes applied data packets containing 
routing tag signals indicative of the output port 
destination to which the data packets are to be ap- 
plied. The packet switching node comprises a plurality 

10 of input ports and output ports with a multiport memory 
coupled therebetween. The memory has a predetermined 
number of memory locations available for storage of 
data packets applied to each of the input ports. 
Control logic is coupled to the input and output ports 

15 and the multiport memory which controls the storage of 
data packets in the memory. The control logic also 
controls the routing of the data packets to the output 
ports in accordance with the routing tag signals. 

In general, this invention comprises an N x M 

20 switch node that accepts data packets at any of N input 
ports and routes each to any of H output ports. The 
output selected is determined by the routing tag signal 
in the packet. The control logic is designed so that 
the data packets are effectively sorted according to 

25 their desired output port destination. Arbitration 

logic randomly, in a statistical sense, chooses among 
any data packets that are directed to the same output 
port. The algorithm implemented by the arbitration 
logic is designed so that data packets will not wait 

30 indefinitely to be routed from the switch node. 

However, although both of these nodes improve 
upon the performance and throughput of conventional 
packet switching designs, their performance can be 
improved. An example will illustrate a typical prob- 

35 lem. Assume that processor 1 applies data packets to 
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1 input port 1* Assume that all output ports of the node 
are connected to identical execution units. Processor 
1 assigns routing tag signals to each of the data 
packets transmitted thereby. If processor 1 continu- 

5 ally assigns routing tag signals corresponding to 

output port l r for example, then the remaining output 
ports are not used, and the execution units connected 
thereto are not used. This clearly lessens the effi- 
ciency of the node. 

10 

SUMMARY QF THE INVENTION 
In order to overcome the limitations of prior 
art switching node designs, the present invention 
provides for a load balancing circuit arrangement that 

15 improves the efficiency and performance of existing 

packet switching nodes. In general, the load balancing 
circuit arrangement of the present invention comprises 
a logic circuit for use with a packet switching node 
that processes applied data packets that contain 

2 0 routing tag signals indicative of the output port 

destinations to which the data packets are addressed. 

The load balancing circuit arrangement 
comprises a load balancing circuit coupled to the 
packet switching node which monitors the output port 

25 addresses of the applied data packets. In addition, 

the load balancing circuit monitors the number of data 
packets addressed to each of the output ports. The 
load balancing circuit is adapted to generate new 
routing tag signals identifying output port addresses 

30 which redistribute the output port load. The load 

balancing circuit arrangement includes a tag selection 
circuit coupled to the load balancing circuit and the 
packet switching node which selectively replace? the 
routing tag signals of the applied data packets with 

35 
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1 the new routing tag signals in order to redistribute % 
and balance the output port load. 

One embodiment of the present invention 
provides for a load balancing circuit arrangement for 
5 use with a multiple queue packet switching node that 
distributes data packets to the least full queues. 
This operation re-routes the data packets to output 
ports that are being infrequently used and hence 
provides for more efficient processing. A second 

10 embodiment of the load balancing circuit is provided 

for use with a multiport memory packet switching node. 

In the first embodiment , the switching node 
comprises queue selection logic coupled to a plurality 
of queues which route the applied data packets to the 

15 desired output port destinations. The load balancing 
circuit arrangement for use with the multiple queue 
packet switching embodiment comprises the load balanc- 
ing circuit coupled between a plurality of queues and 
their associated queue selection logic. The load 

20 balancing circuit monitors the number of data packets 
stored in each of the queues. The load balancing 
circuit is coupled to the tag selection circuit which 
provides input signals to the queue selection logic 
that replace the routing tag signals of the data packet 

2 5 being processed with new routing tag signals. The new 
routing tag signals are representative of the queue 
which has the fewest packets stored therein. This 
processing redistributes applied data packets to the 
least full queue and hence redistributes the. output 

30 port loading of the circuit. 

In the second embodiment , the switching node 
comprises a multiport memory and control logic which 
similarly route applied data packets to the desired 
output ports. The load balancing circuit arrangement 

35 for use in the multiport memory packet switching 
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1 embodiment comprises the load balancing circuit coupled 
to the control logic which monitors the number and 
destination of the data packets stored in the multiport 
memory. The load balancing circuit arrangement pro- 
5 vides input signals to the control logic which replace 
the routing tag signals with new routing tag signals 
which redistribute the applied data packets to the 
output ports to achieve balanced output port loading, 
The load balancing circuit arrangement 

10 comprises a load balancing circuit that monitors the 
output port addresses of the applied data packets and 
the number of data packets addressed to each of the 
output ports. This load balancing circuit generates 
the new routing tag signals, A tag selection circuit 

15 coupled to the load balancing circuit selectively 

applys the new routing tag signals to the switching 
node in response to enabling signals applied thereto. 

The load balancing circuit comprises a 
minimum index circuit for generating the new routing 

20 tag signals and an adder circuit coupled to the mimimum 
index circuit for combining the new routing tag signals 
with offset signals which modify the new routing tag 
signals in order to implement a predetermined output 
port priority scheme. 

25 A switching node employing the load balancing 

circuit arrangement of the present invention may be 
used to construct a multistage interconnection networks 
which have several capabilities* The first has the 
ability to distribute incoming data packets over a pool 

30 of shared resources. These resources are typically 
identical execution or memory units connected to the 
output ports of the node. The second employs an extra 
node stage at the input of the network. Enabling the 
load balancing circuit in the switch nodes of the newly 

35 
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1 added stage balances the load throughout the entire 
multistage network. 

Although the present invention is discussed 
with reference to its use with computer systems and 

5 architectures, it is not limited to this application* 
The present invention may also be used in applications 
involving the communications field* In particular, 
interconnection networks employing the present inven- 
tion may be used to connect telephone systems that 
10 communicate both data and voice information by way of 
data packets • 

BRIEF DE5CR TPTI ON OP THE DRAWINGS 

The various objects and features of the 

15 present invention may be more readily understood with 
reference to the following detailed description taken 
in conjunction with the accompanying drawings, wherein 
like reference numerals designate like structural 
elements, and in which: 

20 Pig. 1 illustrates a block diagram of an 

embodiment of a load balancing circuit in accordance 
with the principles of the present invention incorpo- 
rated into one channel of a multiple-queue packet 
switching node; 

25 Pig, 2 illustrates a more detailed block 

diagram of the load balancing circuit of Fig. 1; 

Pigs, 3a and 3b comprise a detailed schematic 
of the logic contained in the minimum index circuit of 
the load balancing circuit of Pig. 2; 

30 Pigs. 4a, 4b and 4c show detailed schematics 

illustrating the logic contained in the modulo four 
adder of the load balancing circuit of Fig, 2 for three 
-priority conditions; 

35 
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1 Pigs* 5a and b comprise a detailed block 

diagram of a multiport memory packet switching node 
without load balancing; 

Fig. 6 illustrates the load balancing circuit 
5 of the present invention coupled to one channel of the 
multiport memory packet switching node of Fig. 5; and 

Fig. 7 illustrates a minimum index circuit 
for use in the load balancing circuit of Fig. 6. 

10 DESCRIPTION OF THE PREFERRED EMBODIMENTS 

Referring to Fig. 1, a block diagram of one 
embodiment of a load balancing circuit arrangement 20 
in accordance with the principles of the present 
invention is shown. The load balancing circuit ar- 

15 rangement 20 is shown incorporated into one channel of 
a multiple queue packet switching node 21. This load 
balancing circuit arrangement 20 is made part of a 
single stage packet switching node 21. The packet 
switching node 21 processes applied data packets 

20 containing routing tag signals indicative of the output 
port destination to which the data packets are ad- 
dressed. 

One channel of the packet switching node 21 
comprises an input port 22 and a plurality of output 

25 ports 26. A queue set 23 is coupled to the input port 
22. The queue set 23 comprises a plurality of queues 
which individually process and store data packets whose 
destinations are a particular one of the output ports 
26* A queue selector 24 is coupled between the input 

30 port 22 and each of the plurality of queues in the 
queue set 23. The queue selector 24 is adapted to 
route applied data packets arriving at the input port 
22 to corresponding ones* of the queues in accordance 
with the addresses contained in the routing tag sig- 

35 nals. 
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1 An output arbitrator 25 is coupled between 

each output port and respective queues of the queue 
set. The output arbitrator 25 processes data packets 
whose destinations are the output port coupled thereto 
5 and applies the data packets stored in each of the 
respective queues to that output port in accordance 
with a predetermined priority arbitration scheme. 

A more detailed description of the basic 
multiple queue packet switching node 21 is provided in 

10 the presently copending application entitled "Packet 
Switched Multiple Queue NxH Switch Node and Processing 
Method, " cited hereinabove. This patent application is 
incorporated herein by this reference. 

The load balancing circuit arrangement 20 is 

15 coupled between each of the queues of the queue set 23 
and the queue selector 24. The load balancing circuit 
arrangement 20 monitors the number of data packets 
stored in each of the queues. The load balancing 
circuit arrangement 20 provides input signals to the 

20 queue selector 24 which replace the routing tag signals 
of the data packets currently being processed. These 
input signals are representative of the queue in the 
queue set 23 which has the fewest packets stored 
therein. 

25 The load balancing circuit arrangement 20 

comprises a load balancing circuit 28 coupled to the 
queues 23 of the packet switching node 20. The load 
balancing circuit 28 monitors the number of data 
packets addressed to each of the output ports 26 by 

30 examining the number of packets in the queue 23 associ- 
ated with each output port 26. The load balancing 
circuit 28 is adapted to generate new routing tag 
signals identifying output pert addresses which redis- 
tribute the output port load. A tag selection circuit 

35 27 is coupled between the load balancing circuit 28 and 
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1 the queue selector 24 which is adapted to selectively 
replace the routing tag signals of the applied data 
packets with the new routing tag signals in order to 
redistribute and balance the output port load. 
5 The load balancing circuit 28 may comprise a 

minimum index circuit coupled to an adder circuit. The 
minimum index circuit is adapted to generate the new 
routing tag signals and the adder circuit is adapted to 
combine the new routing tag signals with offset signals 

10 which modify the new routing tag signals in order to 

implement a predetermined output port priority scheme. 
These two circuits will be described in more detail 
with reference to Fig. 2. The tag selection circuit 27 
may comprise a two to one multiplexer , or the like, 

15 which is adapted to selectively choose between two 
input signals for output to the queue selector 24. 

In operation, the multiple queue packet 
switching node 21 employing the load balancing circuit 
arrangement 20 functions in the following manner. In a 

20 typical multi-processor computer application requiring 
load balancing, or the like, a process controller, or 
the like, is coupled to the input port 22. This 
process controller generates data and instructions 
which are sent to functionally identical execution 

25 units individually coupled to the output ports 26. The 
data and instructions are sent in data packets which 
are processed by the packet switching node 21. The 
switching node 21 provides a communications link 
between the process controller and the execution units. 

30 Each data packet includes a routing tag 

signal identifying the particular output port 26 9 and 
hence the particular execution unit, to which the 
packet is to be sent. In addition, the da^a packet 
also includes data that is to be operated upon in that 

35 execution unit. A data packet enters the switching 
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1 node 21 and the queue selector 22 reads the routing tag 
signal. It then routes the data packet to the appro- 
priate queue of the queue set 23 which processes 
signals whose destinations are the particular output 
5 port 26 corresponding to the routing tag signal. 

In the multi-processor computer of this 
example, each of the output ports are connected to 
functionally identical execution units. Therefore, it 
is not particularly important where the instructions 

10 contained in the data packets get executed, but that 

they are executed in a timely manner. Since all of the 
execution units are functionally identical, routing tag 
signals may be changed in order to more efficiently 
utilize the execution units. 

15 The load balancing circuit arrangement 20 

produces the desired re-routing of the data packets to 
accomplish more efficient operation of the switching 
node 21 and the computer system in general. The load 
balancing circuit 20 processes count signals generated 

20 by each of the queues 23 in the queue set 24. The 
count signals indicate the number of data packets 
stored in a particular queue. The load balancing 
circuit arrangement 20 then analyzes the count signals 
to determine which queue has the least number of data 

25 packets stored therein. Once this has been determined , 
the routing tag signal in the data packet being pro- 
cessed by the queue selection logic 23 is changed to 
identify the least full queue. 

To better understand the operation of the 

30 load balancing circuit 20, reference is made to Fig. 2. 
Pig. 2 shows a more detailed block diagram of the load 
balancing circuit 28. The load balancing circuit 28 is 
comprised of the minimum index circuit 30 and en adder 
circuit 31. In Fig. 2, the load balancing circuit 28 

35 is shown adapted for use with a node which comprises ■ 
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1 four output ports, hence it employs four sets of inputs 
to the minimum index circuit 30, identfied as L[0] to 
L[3]. 

Each set of inputs to the minimum index 
5 circuit 30 is individually coupled to count circuitry 
in the queues which counts the number of data packets 
currently being stored and processed by a particular 
queue. The output of the minimum index circuit 30 is 
coupled to a modulo M adder 31 (modulo 4 in Fig. 2) 
10 which adds an offset to the index produced. The load 
balancing circuit 28 implements an algorithm which 
determines the least full queue in the queue set. A 
typical algorithm is presented below. 



I . INPUT SCALAR: OFFSET 
15 2. tnptjt ABB&X: LtM3 

3. QTITPTTT SCALAR i ADDR 

4. BEGTH 

5. minimum: r0; 

6. £QE U-l; WHILE i<M; ii=i+l) 
20 7. 1£ (L[i] < L[minimum]) 

8. minimum: =i; 

10. ADDR: - (minimum + OFFSET) HQPJ2LQ M; 

II. E33SL 

25 This algorithm scans the inputs L[M] from low 



order to high, looking for the smallest value. In the 
case of a tie, the value is retained, thus implementing 
a particular priority scheme. This priority scheme is 
as follows. Since there can be a tie between two or 
30 more queues for the lowest number, a default priority 
is assigned. Let L[0] be the first input to the 
minimum index circuit 30 and let L[M-1] be the last 
input. In the case of a tie, LtOJ has the highest 
priority and L[H-1] has the lowest. 

35 
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1 Alternatively, if the first queue is not to 

have the highest priority, rather queue i, 0<i<M, then 
the queue i data packet c.-unt is connected to MO], the 
queue with the next highest priority is connected to 
5 L[l], and so on. In addition, in order to obtain the 
correct re-routing tag signal, the binary representa- 
tion of i is connected to the OFFSET input to the 
modulo M adder 31. To convert this algorithm into a 
logic circuit requires the specification of the maximum 

10 queue size and the number of output ports (M) , in order 
to determine the number of input and output bits in the 
load balancing circuit 28. 

The ability to specify priority is important. 
In an NxM packet switching node, for example, N of the 

15 load balancing circuits are required. If all circuits 
gave top priority to the lower numbered outputs, there 
would be a bias in the overall operation of of the 
switching node favoring these outputs. The distribu- 
tion of the load over the outputs would not be uniform. 

20 To illustrate how the load balancing circuit 

28 is reduced to logic, assume a four by four packet 
switching node with queues that can hold up to three 
data packets. Therefore, the number of input ports N = 
4 and the number of output ports M « 4, and the number 

25 of data packets P = 3. The equation pLog 2 {P+in 58 

rLog 2 {3+l)~l - 2 represents the number of bits required 
to indicate the data packet count in any queue. Since 
there are four queues, a total of eight inputs are 
needed for the minimum index circuit 30, Also, flog 2 Ml 

30 « 2 bits are required to indicate the which queue is to 
be selected. Therefore, each input to the minimum 
index circuit 30, represented by L[i] , is represented 
by two bits, Lil and LiO, and ADDS and OFFSET are 
represented by two bits each, ADDRl, ADDR0, and OFF- 

35 SET1, OFFSET0, respectively. 
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1 The load balancing circuit 28 comprises two 

parts, the minimum index circuit 30 and the modulo four 
adder 31 which combines the output of the minimum index 
circuit 30 with the OFFSET value. A modified algorithm 

5 is required to implement the minimum index circuit 
only* and is presented below. 

1. I NP UT L[Ml 

2. OUTPUT SCALAR; OUT 

3. BEGIN 

10 4. minimum:=0; 

5. EQR HHXLE i<M; i:»i+l) 

6. IE CL[i] < Llminimum]) 

7. minimum: «i; 

8. SNPFPR 

15 9. OUT:«minimum; 

10. £ED 

This algorithm has been programmed into a 
computer to generate a truth table with eight boolean 
input variables and two boolean output variables. This 
20 truth table was then made the input to a program which 
generated the following "eum-of-product" form logic 
equations. These equations correspond directly to a 
two level nand gate implementation of the minimum index 
circuit 30. 

25 OUT1 - lTL-lTo .L1CL00 + L21 • L20 -L10 «L01 

+ L21«L20 'L11-L00 + L31 * Lll • L01 
+ L31-L3$-L10»L01 + L3"l* L30 • Lll * LOO 
+ L21 'L2u»L10-LO0 + l5i»lii*loi 
+ lTQ-LII -LlO'LOl-LOO 

30 + E2T5-L11-L10-L01 'L00 

outo * lTI'lTo.loo + LTI-lTO'LOI 

+ L31-L30.L2O.L00 + L20*LlT*L01 
+ L31-L3"b'L20-l01 + lTI*L30 -L21-L00 
+ L3l*L21.L01 + L21-lTi»L01 
35 + L21»L10 «L01 »L00 + L30 -L21 * L20 'L01 «L00 
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1 A detailed schematic of the logic diagram 

representing a specific embodiment of the minimum index 
circuit 30 is shown in Pig. 3. Pig. 3 is comprised of 
Pigs. 3a ana 3b. Pigs. 3a and 3b should be placed 
5 abutting one another and joined along the line of 
separation in order to obtain the complete drawing. 

The "sum-of-product* form of the equations 
for the modulo four adder 31 are as follows. 

CI m B1»A1*A0 + Bl-BO-Al <+ B1»B0-A1 

10 + B1.A1*a5 + bI*B0*A1-A0 + Bl*B0*Al*A0 

CO m BO *A0 + BO • AO - B ® A 
Since the B input is the OFFSET constant, once the 
constant is selected, it is substituted into the above 
equations which results in simpler circuits. If the 

15 OFFSET m 0, no circuit is required. Figs. 4a, 4b and 

4c show the circuits to be used when the OFFSET is 1, 2 
and 3, respectively. A detailed discussion of the 
components and wiring of the circuit of Figs. 3 and 4 
will be dispensed with. 

20 The load balancing circuit arrangement of the 

present invention may also be incorporated into a 
multiport memory packet switching node. Fig* 5 illus- 
trates a portion of a four input, four output multiport 
memory packet switching node without load balancing. 

25 This circuit is identical to the circuit of Fig. 3 of 
the presently copending multiport memory patent appli- 
cation cited hereinabove. Fig, 5 shows control logic 
40 of the multiport memory packet switching node. The 
control logic comprises four input port logic sections 

30 41a-d coupled to four output port logic sections 42a-d. 
In addition, handshaking signal lines 45 and status 
update control logic 46 are included in Fig. 5. 

Referring to Pig. 6, there is shown a portion 
of one channel of the multiport memory packet switching 



BNSDOCID: <WO 060251 1A1 f > 



WO 86/0251 1 PCT/US85/01839 



17 

1 node of Fig. 5 incorporating a load balancing circuit 
arrangement 20' of the present invention • The discus- 
sion of this embodiment is undertaken with reference to 
the presently copending patent application entitled 
5 "Packet Switched Hultiport Memory NxM Switch Node and 
Processing Method" cited hereinabove with particular 
reference to Fig. 3 thereof. 

Fig. 6 shows a portion of the control logic 
40 which includes an input port logic section 41a 

10 coupled to four output port logic sections 42a~d. A 
more detailed understanding of the complete circuit 
will be had with reference to Fig. 3 in the previously 
cited patent application. The signal lines illustrated 
in Fig. 6 of this application correspond to signal 

15 lines described in Fig. 3 of the copending patent 
application. 

The load balancing circuit arrangement 20 1 
has inputs coupled to request output lines REQ0, REQ1, 
REQ2 and REQ3 of the input port logic section 41. 

20 Outputs of the load balancing circuit arrangement 20 f 
are coupled to a latch 45 which is used to ensure 
correct signal timing and then to the enabling circuit 
27. A load balancing circuit 28 1 comprises a minimum 
index circuit 30 f and the adder 31. The minimum index 

25 * circuit 30 1 for use in this embodiment is shown in more 
detail in Fig. 7. The adder circuit 31 is substantial- 
ly the same as hereinabove described with reference to 
the load balancing circuit arrangement for the multiple 
queue node. 

30 Thus, there has been described a load balanc- 

ing circuit for use with a packet switching node which 
may be employed in multi-processor and parallel comput- 
er systems, and the like. The load balancing circuit 
may be used to automatically redistribute data packets 

35 to external execution units coupled to a multiple queue 
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1 or multiport memory switching node. This redistribu- 
tion results in more efficient operation of the node 
and computer system employing networks incorporating 
the load balancing circuit. 
5 It is to be understood that the above- 

described embodiments are merely illustrative of some 
of the many specific embodiments which represent 
applications of the principles of the present inven- 
tion. Clearly, numerous and varied other arrangements 

10 may be readily devised by those skilled in the art 
without departing from the spirit and scope of the 
invention. In particular, the present invention may 
also be used in applications involving the communi- 
cations field. Interconnection networks employing the 

15 principles of the present invention may be used to 

connect telephone systems that communicate both data 
and voice information by way of data packets. 

20 



25 



30 



35 
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What is claimed is: 

1 1. A load balancing circuit arrangement for 

use with a packet switching node that processes applied 
data packets that contain routing tag signals indica- 
tive of the output port destinations to which said data 
5 packets are addressed, wherein said load balancing 
arrangement comprises: 

load balancing means coupled to said packet 
switching node for monitoring the output port addresses 
of said applied data packets and the number of data 
10 packets addressed to each of said output ports, and for 
generating new routing tag signals identifying output 
port addresses which are adapted to redistribute the 
output port load; and 

tag selection means coupled to said load 
15 balancing means and said packet switching node for 

selectively replacing the routing tag signals of said 
applied data packets with said new routing tag signals 
in order to redistribute and balance the output port 
load. 

1 2. The load balancing circuit arrangement 

of Claim 1 wherein: 

said load balancing means comprises a logic 
circuit which monitors the output port addresses of 
5 said applied data packets and the number of data 

packets addressed to each of said output ports, and 
generates said new routing tag signals; and 

said tag selection means comprises a logic 
circuit for selectively applying said new routing tag 
10 signals to said switching node in response to enabling 
signals applied thereto. 
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3. The load balancing circuit arrangement 
of Claim 2 wherein said load balancing means comprises: 

a minimum index circuit for generating said 
new routing tag signals; and 

an adder circuit coupled to said minimum 
index circuit for combining said new routing tag 
signals with offset signals which modify said new 
routing tag signals in order to implement a predeter- 
mined output port priority scheme*. 

4. A load balancing circuit arrangement for 
use with a switching node that processes applied data 
packets containing routing tag signals indicative of 
the output port destinations to which said data packets 
are addressed , and wherein said switching node compris- 
es a multiport memory and control logic coupled thereto 
for routing applied data packets in accordance with the 
output port destination thereof, wherein said load 
balancing circuit arrangement comprises: 

load balancing means coupled to said control 
logic for monitoring the output port addresses of said 
applied data packets and the number of data packets 
addressed to each of said output ports , and for gener- 
ating new routing tag signals identifying output port 
15 addresses which are adapted to redistribute the output 

port load; and 

tag selection means coupled to said load 
balancing means and said control logic for selectively 
replacing the routing tag signals of said applied data 
packets with said new routing tag signals in order to 
redistribute and balance the output port load. 

5. The load balancing circuit arrangement 
of Claim 4 wherein: 



10 



20 
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said load balancing means comprises a logic 
circuit which monitors the output port addresses of 
5 said applied data packets and the number of data 

packets addressed to each of said output ports, and 
generates said new routing tag signals? and 

said tag selection means comprises a logic 
circuit for selectively applying said new routing tag 
10 signals to said switching node in response to enabling 

signals applied thereto. 

1 6. The load balancing arrangement of Claim 

5 wherein said load balancing means comprises: 

a minimum index circuit for generating said 
new routing tag signals; and 

5 an adder circuit coupled to said mimimum 

index circuit for combining said new routing tag 
signals with offset signals which modify said new 
routing tag signals in order to implement a predeter- 
mined output port priority scheme. 

1 7. A load balancing circuit arrangement for 

use with a packet switching node that processes applied 
data packets containing routing tag signals indicative 
of the output port destinations to which said data 
5 packets are addressed r and wherein said switching node 

comprises queue selection logic coupled to a plurality 
of queues for routing applied data packets in accor- 
dance with the output port destination thereof , wherein 
said load balancing circuit arrangement comprises: 

10 load balancing means coupled to said plurali- 

ty of queues for monitoring the number of data packets 
addressed to each of said output ports, and for gener- 
ating new routing tag signals identifying output port 
addresses which are adapted to redistribute the output 

15 port load; and 
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tag selection means coupled to said load 
balancing means and said queue selection logic for 
selectively replacing the routing tag signals of said 
applied data packets with said new routing tag signals 
in order to redistribute and balance the output port 
load* 

8. The load balancing circuit arrangement 
of Claim 7 wherein: 

said load balancing means comprises a logic 
circuit which monitors the output port addresses of 
said applied data packets and the number of data 
packets addressed to each of said output ports, and 
generates said new routing tag signals; and 

said tag selection means comprises a logic 
circuit for selectively applying said new routing tag 
signals to said switching node in response to enabling 
signals applied thereto. 

9. The load balancing circuit arrangement 
of Claim 8 wherein said load balancing means comprises 

a minimum index circuit for generating said 
new routing tag signals; and 

an adder circuit coupled to said minimum 
index circuit for combining said new routing tag 
signals with offset signals which modify said new 
routing tag signals in order to implement a predeter- 
mined output port priority scheme. 
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